%pip install jupyter-book
!rm -rf ./jb && wget -q -O jb.zip https://github.com/ouseful-testing/jupyter-book-server-proxy/zipball/main/ && unzip -d ./jb jb.zip && rm jb.zip
!pip install ./jb/ouseful-testing-jupyter-book-server-proxy-a84754a/
NOW RESTART THE DOCKER CONTAINER
%%writefile _config.yml
title: TM351 Notebooks
auther: TM351 Module Team, The Open University
only_build_toc_files : true
exclude_patterns : [_build, Thumbs.db, .DS_Store, "**.ipynb_checkpoints", "**.md", "**jupyter_book_server_proxy"]
execute:
execute_notebooks : off
launch_buttons:
jupyterhub_url : http://localhost:8351/notebooks/notebooks # The URL of the JupyterHub (e.g., https://datahub.berkeley.edu)
thebe : true # Add a thebe button to pages (requires the repository to run on Binder)
You can create a table of contents (_toc.yml
) file from the .ipynb
file contents of the current directory (and subdirectories) therefrom using a command such as jupyter-book toc from-project . -e .ipynb -f jb-book > _toc.yml
.
The file may need fettling to get the structure you require (about).
The following example is not ideal (generated from TM351 notebooks).
%%writefile _toc.yml
format: jb-book
root: Using Jupyter Notebooks - READ ME FIRST
chapters:
- file: Part 01 Notebooks/01.1 Getting started with IPython and Jupyter Notebooks
- Bootcamp
sections:
- file: Part 01 Notebooks/01.2 Python recap
- file: Part 01 Notebooks/01.3 Basic python data structures
- file: Part 01 Notebooks/01.4 Defining new functions in python
- file: Part 01 Notebooks/01.5 Python file handling
- file: Part 01 Notebooks/01.X Customising the Notebook Environment
- file: Part 02 Notebooks/02.1 Pandas Dataframes
sections:
- file: Part 02 Notebooks/02.2 Data file formats
- file: Part 02 Notebooks/02.2.0 Data file formats - file encodings
- file: Part 02 Notebooks/02.2.1 Data file formats - CSV
- file: Part 02 Notebooks/02.2.2 Data file formats - JSON
- file: Part 02 Notebooks/02.2.3 Data file formats - other
- file: Part 03 Notebooks/03.1 Cleaning data
sections:
- file: Part 03 Notebooks/03.2 Selecting and projecting, sorting and limiting
- file: Part 03 Notebooks/03.3 Combining data from multiple datasets
- file: Part 03 Notebooks/03.4 Handling missing data
- file: Part 04 Notebooks/04.1 Crosstabs and pivot tables
sections:
- file: Part 04 Notebooks/04.2 Descriptive statistics in pandas
- file: Part 04 Notebooks/04.3 Simple visualisations in pandas
- file: Part 04 Notebooks/04.4 Activity 4.4 Walkthrough
- file: Part 04 Notebooks/04.5 Split-apply-combine with SQL and pandas
- file: Part 04 Notebooks/04.5.soln SalesTeamExploration
- file: Part 04 Notebooks/04.6 Introducing regular expressions
- file: Part 04 Notebooks/04.7 Reshaping data with pandas
- file: Part 05 Notebooks/05.1 Anscombe's Quartet - visualising data
sections:
- file: Part 05 Notebooks/05.2 Getting started with maps - folium
- file: Part 05 Notebooks/05.3 Getting started with matplotlib
- file: Part 07 Notebooks/07.1 Spreadsheet basics
sections:
- file: Part 07 Notebooks/07.2 Problems in spreadsheet construction
- file: Part 08 Notebooks/08.1 Data Definition Language in SQL
sections:
- file: Part 08 Notebooks/08.2 Data Manipulation Language in SQL
- file: Part 08 Notebooks/08.3 Adding column constraints to tables
- file: Part 08 Notebooks/reset_databases
- file: Part 08 Notebooks/sql_init
- file: Part 09 Notebooks/09.1 Defining Foreign Keys in SQL
sections:
- file: Part 09 Notebooks/09.2 Using foreign keys in SQL
- file: Part 09 Notebooks/09.3 Working With FOREIGN KEY Constraints
- file: Part 09 Notebooks/reset_databases
- file: Part 09 Notebooks/sql_init
- file: Part 10 Notebooks/10.1 problems with unnormalised data
sections:
- file: Part 10 Notebooks/10.2 Normalisation - Antique opticals
- file: Part 10 Notebooks/10.3 Normalisation - the Hospital scenario
- file: Part 10 Notebooks/10.4 Our solution to Normalisation - the Hospital scenario
- file: Part 10 Notebooks/10.5 Improvements with normalised data
- file: Part 10 Notebooks/reset_databases
- file: Part 10 Notebooks/sql_init
- file: Part 11 Notebooks/11.0 Setting up the Movie database
sections:
- file: Part 11 Notebooks/11.1 Movie analysis
- file: Part 11 Notebooks/11.2 subqueries as value and set
- file: Part 11 Notebooks/11.3 Subqueries as tables
- file: Part 11 Notebooks/11.4 Views
- file: Part 11 Notebooks/11.5 Six degrees of Bacon
- file: Part 11 Notebooks/reset_databases
- file: Part 11 Notebooks/SQL_cheatsheet
- file: Part 11 Notebooks/sql_init
- file: Part 12 Notebooks/12.1 Concurrent Transactions
sections:
- file: Part 12 Notebooks/12.2 Transaction anomalies
- file: Part 12 Notebooks/reset_databases
- file: Part 12 Notebooks/sql_init
- file: Part 12 Notebooks/optional_part_12/12.3 Optional- Concurrent Transactions
and Multiple Threads
sections:
- file: Part 12 Notebooks/optional_part_12/12.3a Gibson
- file: Part 12 Notebooks/optional_part_12/12.3a Paxton
- file: Part 12 Notebooks/optional_part_12/reset_databases
- file: Part 12 Notebooks/optional_part_12/sql_init
- file: Part 14 Notebooks/14.1 Basic CRUD
sections:
- file: Part 14 Notebooks/14.2 Working With Embedded Documents
- file: Part 14 Notebooks/14.3 Importing Data into MongoDB
- file: Part 14 Notebooks/14.4 Introduction to the accidents database
- file: Part 14 Notebooks/14.5 Investigating the accident data
- file: Part 14 Notebooks/14.A Optional- Using statistical tests - correlation
- file: Part 14 Notebooks/14.B Optional- Using statistical tests - regression
- file: Part 15 Notebooks/15.1 Mapping accidents
sections:
- file: Part 15 Notebooks/15.2 Searching within a geographical area
- file: Part 15 Notebooks/15.3 Introducing aggregation pipelines
- file: Part 15 Notebooks/15.4 Grouping and summarising operations in aggregation
pipelines
- file: Part 15 Notebooks/15.5 Introducing the Roads collection
- file: Part 15 Notebooks/15.6 Working with roads location data
- file: Part 16 Notebooks/16.1 Accidents over time
sections:
- file: Part 16 Notebooks/16.2 Python map-reduce
- file: Part 16 Notebooks/16.3 Introducing map-reduce in MongoDB Database Queries
- file: Part 20 Notebooks/20.1 The k-nearest neighbours classifier
sections:
- file: Part 20 Notebooks/20.2 The leave-one-out algorithm
- file: Part 21 Notebooks/21.1 The k-means algorithm
sections:
- file: Part 21 Notebooks/21.2 k-means clustering - choosing initial values
- file: Part 21 Notebooks/21.3 Visualising clusters with silhouette coefficients
- file: Part 22 Notebooks/22.1 Case study preliminaries - the vector space model
sections:
- file: Part 22 Notebooks/22.2 Preliminaries - building the classifier
- file: Part 22 Notebooks/22.3 Applying the classifier to a real dataset
- file: Part 22 Notebooks/22.4 Term frequency and inverse document frequency
- file: Part 23 Notebooks/23.1 SQL injection hacks
sections:
- file: Part 23 Notebooks/form_server
- file: Part 23 Notebooks/form_server_safe
- file: Part 23 Notebooks/part_23_authentication_notebook
- file: Part 23 Notebooks/reset_form_server
- file: Part 25 Notebooks/25.1 Exploring the RDFLib package
sections:
- file: Part 25 Notebooks/25.2 Querying using SPARQL
- file: Part 25 Notebooks/25.3 Endpoints - accessing real data
- file: Part 26 Notebooks/26.1 Using multiple endpoints
sections:
- file: Part 26 Notebooks/26.2 The SPARQL CONSTRUCT query and inferencing
- file: Part 26 Notebooks/26.3 Visualisation
replace_jh_from = 'hub/user-redirect/git-pull?repo=https://github.com/executablebooks/jupyter-book&urlpath=tree/jupyter-book/'
replace_jh_with = ''
replace_jh_from2 = '&branch=master"><button type="button"'
replace_jh_with2 = '"><button type="button"'
replace_jh_from3 = 'alt="Interact on JupyterHub">JupyterHub</button>'
replace_jh_with3 = 'alt="Interact on local notebook server">localhost</button></a>'
replace_thebe_from = '''<script type="text/x-thebe-config">'''
replace_thebe_with='''<script type="text/x-thebe-config">
{
requestKernel: true,
kernelOptions: {
name: "python3",
serverSettings: {
"baseUrl": "http://localhost:8351",
"token": "letmein"
}
},
}
</script>
<!--
'''
replace_thebe_from2 = '''<script>kernelName = 'python3'</script>'''
replace_thebe_with2 = '''-->
<script>kernelName = 'python3'</script>'''
The Thebe update actually wipes out the start directory that appears in the orignal as eg:
kernelOptions: {
kernelName: "python3",
path: "./Part 16 Notebooks"
},
I'm not sure if the path can be set for the local usage (not tried). It would be really handy if it could...
# Generate book
!jupyter book build --path-output . .
# Files in _build/html
# Update files
from pathlib import Path
html_dir = Path('./_build/html')
for p in html_dir.rglob("*"):
if p.is_file() and p.suffix == '.html':
# Read file
#print(p)
with p.open('r', encoding ="utf-8") as f:
html = f.read()
html = html.replace(replace_jh_from, replace_jh_with)
html = html.replace(replace_jh_from2, replace_jh_with2)
html = html.replace(replace_jh_from3, replace_jh_with3)
html = html.replace(replace_thebe_from, replace_thebe_with)
html = html.replace(replace_thebe_from2, replace_thebe_with2)
with p.open("w", encoding ="utf-8") as f:
f.write(html)
!rm -rf ./jb/ouseful-testing-jupyter-book-server-proxy-a84754a/jupyter_book_server_proxy/static/
!mkdir -p ./jb/ouseful-testing-jupyter-book-server-proxy-a84754a/jupyter_book_server_proxy/static/
!cp -r _build/html/* ./jb/ouseful-testing-jupyter-book-server-proxy-a84754a/jupyter_book_server_proxy/static/
!pip install ./jb/ouseful-testing-jupyter-book-server-proxy-a84754a/
FROM THE NOTEBOOK HOMEPAGE, REFRESH THE PAGE, THEN SELECT New > Jupyter Book
If the menu item is not there, restart the container then reload the home page.
Have a play... From the rocket menu you should be able to navigate to the notebooks as well as enabling code execution in page (Live Code).
Note that at the moment the file paths to load files are broken. The kernel is in running from /home/jovyan
so you need to make paths relative to that. There is a way to set the kernel start directory when running using MyBinder, but I'm not sure offhand if that works at the moment with using ThebeLab to run locally (I will explore this when I get back from hols).